Sourced from intel-extension-for-pytorch's releases.

Intel® Extension for PyTorch* v2.3.100+cpu Release Notes

Highlights

• Added the optimization for Phi-3: #2883

• Fixed the state_dict method patched by ipex.optimize to support DistributedDataParallel #2910

• Fixed the linking issue in CPPSDK #2911

• Fixed the ROPE kernel for cases where the batch size is larger than one #2928

• Upgraded deepspeed to v0.14.3 to include the support for Phi-3 #2985

Full Changelog: intel/intel-extension-for-pytorch@v2.3.0+cpu...v2.3.100+cpu

Intel® Extension for PyTorch* v2.3.0+cpu Release Notes

We are excited to announce the release of Intel® Extension for PyTorch* 2.3.0+cpu which accompanies PyTorch 2.3. This release mainly brings you the new feature on Large Language Model (LLM) called module level LLM optimization API, which provides module level optimizations for commonly used LLM modules and functionalities, and targets to optimize customized LLM modeling for scenarios like private models, self-customized models, LLM serving frameworks, etc. This release also extends the list of optimized LLM models to a broader level and includes a set of bug fixing and small optimizations. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try this release and feedback as to improve further on this product.

Highlights

• Large Language Model (LLM) optimization

  Intel® Extension for PyTorch* provides a new feature called module level LLM optimization API, which provides module level optimizations for commonly used LLM modules and functionalities. LLM creators can then use this new API set to replace related parts in models by themselves, with which to reach peak performance.

  There are 3 categories of module level LLM optimization APIs in general:

  • Linear post-op APIs

  # using module init and forward
  ipex.llm.modules.linearMul
  ipex.llm.modules.linearGelu
  ipex.llm.modules.linearNewGelu
  ipex.llm.modules.linearAdd
  ipex.llm.modules.linearAddAdd
  ipex.llm.modules.linearSilu
  ipex.llm.modules.linearSiluMul
  ipex.llm.modules.linear2SiluMul
  ipex.llm.modules.linearRelu

  • Attention related APIs

  # using module init and forward
  ipex.llm.modules.RotaryEmbedding
  ipex.llm.modules.RMSNorm
  ipex.llm.modules.FastLayerNorm
  ipex.llm.modules.VarlenAttention
  ipex.llm.modules.PagedAttention

... (truncated)

• See full diff in compare view

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)